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Abstract 

The present article introduces the Fuzzy Delphi method results obtained in the study on determining e-Portfolio 
elements in learning process for art and design context. This method bases on qualified experts that assure the 
validity of the collected information. In particular, the confirmation of elements is based on experts’ opinion and 
consensus. The consensus survey constructed based on the emergent themes the experts raised from the 
conducted interview. For this purpose about 23 experts in instructional technology involved in the interview and 
responses the survey. As resulted, the Fuzzy Delphi will interpret the decision making made by experts based on 
priority as a guideline to the best practices and mechanism of implementing e-Portfolio as methodological tool. 
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1. Introduction 

Constructivism paradigm emphasizes on the individual in constructing their own knowledge. Knowledge and 
skills growth based on the subject changes and the modification of learning context. Students use e-Portfolio is 
to demonstrate their concept, understanding and skills required to fulfill the learning outcome. Respectively, the 
uses of e-Portfolio are to improve students’ learning, develop a personalities, growth and autonomy in learning in 
art and design courses. 

Commonly, art and design assessment practice implemented evidence-based to prove learning process during the 
course. Therefore, art and design current interest is using e-Portfolios of students has been driven by the 
Malaysian Qualification Accreditation (MQA) as a standard practices. Hence, methods of student learning and 
assessment have to be clear, consistent, effective, reliable and in line with current practices and must clearly 
support the achievement of learning outcomes (COPPA2008). 

The MQA has been a benchmark for the proportion of learning and assessment, which can be adapted by art and 
design program in higher education institution. Curriculum Affairs Unit of Universiti Teknologi MARA (2010) 
has reported that 40-70% of continuous assessment is the common practice being used and implemented in many 
faculties. They were also applying the use of portfolio as a mechanism of evidence-based assessment to access 
students’ performance and competencies. 

The e-Portfolio is seems useful not only as a learning tool but also useful as an archive of learning. The building 
of content knowledge and skills was documented as a preferal for future references. At the end of the day, the 
students will be equipped with a specific learning domain that enable them to a professional or expert in 
respective area. This paper establishes the e-Portfolio elements to improve teaching and learning process. 

1.1 E-Portfolio Learning Process 

The e-Portfolio derives from so many researchers opinion, which is a process to develop and access personal and 
professional activities. The main difference between traditional and e-portfolio is the ease to publish contents and 
evidence in electronic format. 

The use of e-portfolio is promoting a students to be a learner-centered and authentic (Read & Cafolla, 1999). 
This process allows students to have opportunities to reflect upon their learning and instructor to provide the 
feedback (Ahn, 2004). Hence, the usefulness of e-portfolio is able to assess students competences in the terms of 
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evaluating this new method’s as efficiency formative tool. The students have to prove certain set of skills 
involves, problem-solving, critical thinking, reflective learning, analyzing and synthesizing (Clark & Eynon 
2009; Chin-Hung, 2013). 

So far, some researchers such (Lynch & Purnawarman, 2004: Beck et al., 2005; Klenowski et ah, 2007), have 
found 2 factors that show the importance of e-portfolio elements could become a support tool for teaching and 
learning. The factors describes, as: 1) It enhances learning through a domain of learning and 2) The alignment of 
academic course outcome with a specific standard of teaching, learning and assessment. 

What is so significant about evidence-based that putting in learning process together with the e-Portfolio? The 
e-Portfolio were designed based on the structure that driven by the philosophical and pedagogical goals. The 
alignment of instructional objective emphasized students processes, articulated and judgment learning through 
their reflective practice as so called as evidence-based. The establishment of reflective practice that helps 
students accumulate the knowledge to constructs meaningful learning such to test their beliefs, intentions and 
actions. 

Every constructs of meaningful learning engages with the inquiry into what they have learned by researching and 
identifying the context or body of knowledge and skills. Students were able to create content, reflect, collaborate 
and communicate on their creative process to demonstrate the art-making process. The diagnostic of learner 
competencies that based on e-Portfolio learning process allows students to understand their strength, weakness 
and opportunity to improve their standard and quality from time to time. 

On the other hand, it has been identified a gap, there hardly exist any publication with empirical result showing 
the elements related to art and design of this methodological tool. Therefore, this paper provides the experts to 
confirm and validate the elements of e-portfolio within teaching and learning of students’ competences. In the 
following sections, we will describe the experts defining the element of e-portfolio as a learning process focused 
on art and design context. 

2. Methodology 

2.1 Participant 

Fuzzy Delphi Method (FDM) was derived to solve the problem of traditional Delphi method (Ishikawa et al., 
1993). This method bases on group thinking of the qualified experts that assures the validity of the collected 
information. The first phase of data collection involved semi-structured interview with seven (7) experts in 
e-Portfolio based in the different universities. The interview protocol was prepared beforehand. 

The emerging themes identified form the interviewers were used for develop the “E-Portfolio Consensus 
Survey”. Second phase, the “E-Portfolio Consensus Survey” is to get the agreement based on the themes given 
by the experts. The consensus survey will be using the 5 point Likert scale anchored in strongly disagree and 
strongly agree. Experts will be required to indicate the extent of their agreement with the statements given. It 
will be constructed based on the emergent themes the experts raised from the conducted interview in the 
previous phase. For this purpose about 16 experts in instructional technology involved and responses the survey. 
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Figure 1. Fuzzy Delphi method 
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2.2 Data Analysis 

Step 1: Assume that K experts are invited to determine the importance of the evaluation criteria and the ratings of 
alternatives with respect to various criteria using variables (Table 1). 

Step 2: Convert the variables into triangular fuzzy numbers as suggested in Tables 1. Let fuzzy numbers be the 
rating of alternative i with respect to criteria ro^and be the j th criteria weight of the kth expert for i= 1 , , to , 

y=l,..., n, k=l ,..., K. and r tj = £(+ r y r 2i; ± r kij ) 

Table 1. Variable for the importance weight of criteria 


Variable 

Fuzzy Scale 

Strongly disagree 

(0.0, 0.1, 0.2) 

Disagree 

(0.1, 0.2, 0.4) 

Not Sure 

(0.2, 0.4, 0.6) 

Agree 

(0.4, 0.6, 0.8) 

Strongly Agree 

(0.6, 0.8, 1.0) 


Step 3.-For each expert, use the vertex method to compute the distance between the average ~rij and r,y and the 
distance between the average nr^-and mf, k =1, K (Chen, 2000). The distance between two fuzzy numbers 
m = (ml, m2, m3) andff = (nl,n2, n3)is computed by, 

d(mn ) = [(ml — nl)2 + (m2 — n2)2 + (m3 — m3)2] 

Cheng and Lin (2002) suggested, if the distance between the average and expert’s evaluation data is less than the 
threshold value of 0.2, then all experts are considered to have achieved a consensus. Furthermore, among those 
to x n ratings of alternatives and n criteria weights, if the percentage of achieving a group consensus is greater 
than 75% (Chu, 2008; Murry & Flammons, 1995) then proceed to step 4; or otherwise, the second round of 
survey is required. 

Step 4: Aggregate the fuzzy evaluations by, 


A 


A1 

i42 


where ArX w t + r i2 X w 2 + 


\Am 


wr in x w n 


i —1,2, . ,m 


Step 5: For each alternative option, the fuzzy evaluation 
A = (ail, ai2, at3) is defuzzified by 
A — -ail + 2ai2 + 2ai3 

4 

The ranking order of alternative options can be determined according to the values of ai 

3. Findings and Discussion 

Table 2 show that the e-Portfolio as a learning tool has consensus among expert with threshold score below then 
0.2. This finding shows that the findings are suited to first rules with threshold score (d) < 0.2. Second rules in 
Fuzzy Delphi also accepted whereas percentage consensus of expert more than 75% expert agreed. 
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Table 2. Expert consensus about e-Portfolio elements based on threshold (d) value 

ITEMS 


Experts 

a part of the course 

aligned to specific 
standards 

deep learning 

authentic 

evidence-based 

Metacognitive 

skills 

think critically 

in-depth learning 

reflect learning 

think differently 

quality 

responsibility as 
owners. 

purpose of doing 
things. 

freedom to control 
the contents 

1 

0.2 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.1 

0.1 

0.4 

0.1 

0.2 

0.1 

0.1 

2 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.4 

0.1 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

3 

0.1 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

0.1 

0.2 

0.1 

0.2 

0.2 

0.2 

0.2 

4 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

5 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

6 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

7 

0.1 

0.1 

0.1 

0.2 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

0.1 

0.2 

0.1 

0.1 

8 

0.1 

0.1 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

0.2 

0.1 

0.7 

0.4 

0.7 

0.7 

9 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

10 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

0.2 

11 

0.2 

0.2 

0.2 

0.1 

0.2 

0.2 

0.2 

0.1 

0.4 

0.1 

0.1 

1.0 

0.4 

0.2 

12 

0.2 

0.2 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.2 

13 

0.1 

0.1 

0.1 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

14 

0.1 

0.1 

0.4 

0.2 

0.2 

0.2 

0.1 

0.1 

0.1 

0.1 

0.2 

0.2 

0.1 

0.1 

15 

0.7 

0.7 

0.4 

0.2 

0.5 

0.5 

0.1 

0.4 

0.1 

0.1 

0.7 

0.1 

0.1 

0.4 

16 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.4 

0.1 

0.1 

0.1 

0.2 

0.1 

0.1 

0.1 


Table 3. E-Portfolio elements learning Ranking based on score 


Items 

Score/Value 

Average of Fuzzy Number 

Ranking 

a part of the course 

0.663 

11 

aligned to specific standards 

0.675 

5 

deep learning 

0.663 

10 

authentic 

0.713 

2 

evidence-based 

0.725 

1 

metacognitive skills 

0.713 

2 

think critically 

0.650 

12 

in-depth learning 

0.650 

12 

reflect learning 

0.675 

5 

think from differently 

0.663 

7 

quality 

0.663 

7 

responsibility as owners. 

0.679 

4 

purpose of doing things. 

0.650 

12 

freedom to control the contents 

0.663 

7 
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Table 3 shows that evidence based had a highest with consensus expert score 0.725 and in depth learning score 
was a lowest score with 0.650. 

Based on these findings, the elements of e-Porftolio in learning process consist: 

Evidence-based (0.725) was agreed by an experts that it the main focused to have e-Portfolio as a support tool. 
The evidence-based will show the learner’s learning track and progress during the learning process. The 
assessment will use evidence-based to measure the standard and quality of work based on standard rubric refer to 
respective subject or courses. 

Kimball (2005) stated that the authentic is a process of reflection in action or the reiterative process of looking 
the past and future to meet the goal and expectation of their learning. Authentic (0.713) is a key feature of 
e-Portfolio which focuses on real-world and complex problems. The authentic learning develop a cognitive 
domain to think, solve and create by providing engaging activities and supported by a proper scaffolding. 

The authentic task emphasized on the development of metacognitive skills (0.713). These skills allows learners 
concurrently to control, monitor and regulate a different level cognitive process (Flavell, 1979). Individual with a 
greater metacognitive skills are able to control, manage and make adjustment to their learning behaviour (Ford et 
al., 1998). At this juncture, this metacognitive skills leads to higher learning (Dresel & Flaugwitz, 2008). 
Therefore, metacognitive skills is actually apart of national education plan in which every students are prepared 
and equipped with these ability. 

By having e-Porftolio, the learners automatically become as an owners of the content. The ownership is a central 
in e-Portfolio system (Garret, 2011). The beauty of ownership elements where as the learners have a personal 
voice to share and reveal their experience based on the task given with peers. This social learning environment 
will give students more space and motivate them to always connect. This opportunity allows learner with 
instructor and peers have a reflective session (0.675) to share and express the idea and feedback. The best part of 
reflective learning is encourage and make them to think differently (0.663), critical thinking (0.650) and perform 
the best quality of work (0.663) align with the specific standard and outcome (0.675). 

On the other hand, we believes that for the long-terms effect, the learners will gradually develop their in-depth 
learning (0.650) and give them a purpose doing things (0.650) not only to fulfill academic courses but also for 
future carrier as a professional. 
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